A Review of Deep Transfer Learning and Recent Advancements

Deep transfer learning techniques try to tackle the limitations of deep learning, dependency on extensive training data and costs, by reusing obtained knowledge. However, current DTL suffer from either catastrophic forgetting dilemma (losing previously knowledge) or overly biased pre-trained models (harder adapt target data) in finetuning freezing a part model, respectively. Progressive sub-category DTL, reduces effect model case earlier layers adding new layer end frozen model. Even though it has been successful many cases, cannot yet handle distant source data. We propose continual/progressive approach for these limitations. To avoid both biased-model problems, we expand expanding (adding nodes each layer) instead only layers. Hence method is named EXPANSE. Our experimental results confirm that can using this technique. At same time, final still valid data, achieving promising continual approach. Moreover, offer way inspired human education system. termed two-step training: basics first, then complexities uncertainties. The evaluation implies extracts more meaningful features finer basin error surface since achieve better accuracy comparison regular training. EXPANSE (model expansion training) systematic applicable different problems DL models.